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ABSTRACT 

Data from the 1990-91 Tennessee school district 
report cards are used to reexamine some relationships noted in 
previous studies (G. C. Bobbett and others, 1992) and to gain new 
insights because of modifications to Tennessee's report cards from 
1989 to 1991. Report cards now report the Tennessee Comprehensive 
Assessment results and have added more school district 
characteristics, expanding analyses from 8 to 15 variables. Several 
conclusions of the 1988-89 study were reinforced by the current 
investigation. Policymakers at all levels need to consider that few 
of the individual inputs commonly associated in people*s minds with 
the production of student achievement have much impact on student 
performance, and that, with the exception of student attendance and, 
perhaps, per-pupil expenditure, treatment of any isolated variable 
will have little effect. Available evidence suggests that variables 
most worthy of consideration are: (1) organizational culture; (2) 
student motivation; (3) parental involvement; (4) instructional 
methodology; (5) curricular features; and (6) other factors that may 
have a significant influence on student performance. Report cards are 
only as good as the assessment usod to determine student performance. 
The current Tennessee report cards do a reasonable job of reporting 
the status of schools, but they still lack meaningful information on 
which to make decisions for .Improvement. Nine tables present analysis 
results. Four appendixes add information about the variables in table 
form. (SLD) 
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EVALUATION OF THE CATEGORIES CURRENTLY USED IN REPORT CARDS 

WITH STUDENT OUTCOME 

I. INTRODUCTION 

In 1991 , the investigators completed a series of studies of the data reported in 
Tennessee's 1 988-89 school district report cards. In those studies which have been 
reported in several papers (Bobbett, et al., 1992a, 1992b), and at meetings, the 
relationships among eight school district variables (average attendance, average 
professional salaries, county per capita income, expenditure per student, average daily 
membership, percent of oversized classes, percent of students on free or reduced 
lunches and percentage of professional educators on upper Career Ladder levels II and 
III) were examined, and the relationships between each variable and student outcomes 
were determined. 

The study reported herein is an extension of the previous study. In this 
investigation, 1990-91 report card data were used. Because of that, it was possible to 
revisit some of the relationships in the previous study and to gain new insights because 
of modifications in Tennessee's report cards from 1989 to 1991. 

In 1990-91, Tennessee brought "on line" its new Tennessee Comprehensive 
Assessment Program (TCAP), thereby creating a new set of student outcome 
measures. Further, TCAP results were reported in greater detail than previous 
outcome data. Report cards now report TCAP assessment results at substantially 
more grade levels within the school districts than was previously done. 

The 1990-91 report cards also added more school district characteristics; 
thereby enabling the investigators to expand their analyses from 8 to 15 variables. The 
seven added variables include number of schools in the district, percent of enrollment 
change, percent regular diplomas awarded, percent honors diplomas awarded, percent 
vocational students, percent special education students, and percent Chapter I 
students. 

While certain comparisons in the results of the two studies can be made, some 
findings cannot be compared because of the differences in the outcome measures used in 
the different years and because no comparable data were available in certain areas In the 
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Table 1 . Testing Information For Widget City Schools (1 988-89 Report Card Data) 



Testing Information 
for Widget City 


Grade 
Level 


1987-88 


1988-89 


State 
Average 


Basic Sicills First 
Achievement Test 

(percent score) 




3 


90 


88 


80 


Reading 


6 


82 


80 


77 




8 


92 


91 


81 




3 


91 


90 


82 


Math 


6 


67 


71 


66 


8 


77 


84 


66 


Stanford 
Achievement 

(Stanine score) 

7-9 = High 
4-6 » Average 
1 -3 =« Low 


Reading 


2 


6 


7 


6 


5 


6 


6 


5 


7 


6 


6 


5 


Math 


2 


7 


8 


6 


5 


7 


6 


6 


7 


7 


7 


5 


Spelling 


2 


6 


7 


6 


Language 


5 


6 


6 


5 


7 


6 


6 


5 


Environment 


2 


7 


7 


6 


Science 


5 


6 


7 


6 


7 


6 


6 


5 


Listening 


2 


7 


7 


5 


5 


6 


6 


5 


7 


6 


6 


5 


Social 
Science 


5 


6 


6 


5 


7 


6 


6 


5 


Stanford Test of 
Academic Skills 

(TASK 2) 

7-9 = High 
4-6 = Ave^'age 
1-3«Low 


Reading 


9 


6 


6 


5 


12 


6 


6 


5 


Math 


9 


6 


6 


5 


12 


6 


6 


5 


English 


9 


7 


6 


5 


12 


6 


7 


5 


Science 


9 


7 


6 


5 


12 


6 


6 


5 


Social 
Science 


9 


5 


6 


5 


12 


6 


5 


5 


Tennessee Proficiency Test 

(% Students Passing) 


Language 


9 


88 


92 


78 


Math 


9 


95 


98 


90 


Both 


9 


86 


91 


76 
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Table 2. General Information Found In a Typical School District's Report Card; 
1988-89. 



System Information 
for Widget City 


Grade 
Level 


1986-87 


1987-88 


1988-89 


State 
Average 


Number of Schools 




5 


5 


5 


12* 


Average uaiiy iviemDersnip 


K-12 






0,0/ ^ 




% Student Attendance 


K-12 


95.7 


95.3 


95.1 


93.6 


% Enrollment Change 


9-12 


-13.0 


-16.1 


-15,2 


-24.7 


% Oversized Class 


K-12 


1.2 


1.4 


2.3 


3.8 


% of Students on Free or Reduced Price Lunch 


K-12 


23 


21 


21 


42 


Expenditures per pupil 


K-12 


$2,718 


$3,299 


$3,501 


$3,304 


County Per Capita Income 


K-12 


If 


«« 


$12,819 


$12,878 


% Elementary Schools Accredited by SACS 


K-8 




100.0 


100.0 


29.1 


% Secondary Schools Accredited by SACS 


7-12 


100.0 


100.0 


100.U 


64.9 


Professional Educator Information 












% Professionals on Career Ladder Levels 11 & 111 


K-12 


22.9 


21.9 


25.6 


14.8 


Average Professional Salary 




$25,198.60 


$26,085.44 


$30,804.37 


$26,756 


Student Information 










■ 


% Diplomas 
Granted 


Regular 


12 


90.6 


68.7 


75.8 


81.8 


Honors 


12 


49.6 


26.7 


20.0 


8.5 


Special Education 


12 


1.8 


1.4 


1.5 


1.9 


Certificate of Attendance 


12 








0.9 


Seniors not Receiving 
Diploma in Spring 
Graduation 


12 


2.7 


3.2 


2.7 


6.9 


% students in Vocational Education Courses 


7-12 


33.0 


40.9 


41.0 


45.5 


% Students in Special Education 


K-12 


12.1 


11.3 


12.1 


14.2 


% Chapter 1 Students 


K-12 


13.3 


15.5 


12.4 


11.9 



1 988-89 report cards. Tables 1 and 2 present a school district report card as it appeared in 
1988-89, and Tables 3 and 4 represent a 1990-91 school district report card. 

II. METHODOLOGY 

Although the 1990-91 report cards provided test results for grades 2 through 10, 
the investigators organized the data into four levels (elementary, grades 2-5; nniddle, 
grades 6-8; hi^h school, grades 9-10; and systenn) rather than treat each grade level 
separately, grades 2-5 became the elementary level. 

Mean student outcomes (MSOs) were created (by converting reported scores to 
Z scores and computing their means) for each level by combining TCAP data for the 



grades defined within the particular level. In the case of the high school level, the MSO 
was created by combining 1 0th grade TCAP data with the scores reported for the 9th 
grade Tennessee Proficiency (TPT). These MSO were treated as dependent variables, 
as was the case in the analysis of 1988-89 report card data. The 15 school district 
characteristics studied were treated as independent variables that influence student 
outcome. To guide the study seven research questions were developed: 

1 . How do school district characteristics currently reported relate to student 
academic achievement results? 

2. Are there differences in the relationships between dependent and independent 
variables at different school levels (elementary, middle, high school, system)? 

3. How do reported school characteristics relate to each other? 

4. When rank ordered on the basis of student outcomes, how do school districts 
within the state perform in terms of reported school and community 
characteristics? 

5. Do the reported school district characteristics appearjo represent all or most 
factors that influence student academic achievement? 

6. Is there evidence of major change in student academic performance from one 
school level to another within scnool districts? 

7. How might the findings of this study inform educational policy at state and local 
levels? 

Five of the seven questions replicate questions poseci in the previous study. 
Questions 2 and 6 are new questions representing the capacity available in the 1990- 
91 report cards to analyze data at several levels within the school districts and the 
capacity of the current study for comparison with the eariier otudy results. Question 7 is 
a modification of a question posed in the eariier study, because only two test batteries 
(rather than three) were used in the current analysis. 

As in the eariier study, most but not all districts reported comprehensive scores 
on both TCAP and TPT. These districts (1 20) constitute the sample for analysis. 

Twenty school district characteristics were actually reported in the 1990-91 
report cards. In responding to research Question #1 , the investigators first evaluated all 
characteristics to determine their value as independent variables. A KaiserXesX of 
variable sample adequacy was applied to each variable at each level (elementary, 
middle, high school, and system). Five characteristics were eliminated from further 
study: percent elementary schools accredited by SACS, percent high schools 
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Testina Information For Widget City Schools Too (1990-91 Report 
Card Data. 

Widget Too Schools 









Year 


2 


3 


4 


5 


6 


7 


8 


10 


Reading 


State Avq. 


na 


na 


na 


na 


na 


na 


na 


na 


1990-91 


7 


6 


6 


6 


6 


7 


7 


6 


TENNESSEE 


GRADE 


COMPREHENSIVE Language 
ASSESSMENT 

PROGRAM ^TCAP^ r 

Main 


Year 


2 


3 


4 


5 


6 


7 


8 


10 


State Avg. 


na 


na 


na 


na 


na 


na 


na 


na 


1990-91 


7 


6 


6 


6 


6 


6 


7 


6 






Year 


2 


3 


4 


5 


6 


7 


8 


10 


State Avq. 


na 


na 


na 


na 


na 


na 


na 


na 


1990-91 


7 


7 


7 


6 


6 


7 


7 


7 




GR 


ADE 




Year 


2 


3 


4 


5 


6 


7 


8 


10 


State Avq. 


na 


na 


na 


na 


na 


na 


na 


na 


1990-91 


7 


6 


7 


6 


6 


7 


6 


6 


OOUIdi 

Studies 


GRADE 


Year 


2 


3 


4 


5 


6 


7 


8 


10 


State Avg. 


na 


na 


na 


na 


na 


na 


na 


na 


1990-91 


7 


6 


7 


6 


6 


6 


6 


6 








Grade 9 




TENNESSEE Language 


Year 


With SDecial Ed. 


Without Special Ed. 




State Avg. 


na 


na 


1990-91 


90 


91 


PROFICIENCY 

TEST (TPT) Mathematics 






Year 


With Soecial Ed. 


Without Special Ed. 




State Avq. 




na 




1990-91 


98 


98 








Both 


Year 


With SDecial Ed. 


Without Special Ed. 




State Avq. 




na 


1990-91 


88 


90 









Students in Tennessee are given two types of tests. 

Students were introduced this spring to the 
Tennessee Comprohensive Assessment Program 
(TCAP). This program mandates a customized, norm 
referenced and criterion referenced test for grades 2 
through 8, a norm referenced test for grade 10, and the 
Tennessee Proflcfency Test 

The customized test will allow each teacher to 
assess progress of students during the school year with 
a minimum amount of testing time. The program will 
generate consistent types of test scores from grade to 
grade. The norm 'carenced data will allow longitudinal 
status of individual, school, system, and stats growth In 



The criterion referenced data will report the mastery , 
partial mastery, and non-mastery of tested domains for 
each school year. Although the objectives for the 
Tennessee Proficiency Test has been updated, the rules 
and regulations governing the test will remain the same. 

The TennessfiO Proficiency Test measures 
minimum skills in mathematics and language arts. 
Students must achieve a passing score of 70 percent 
correct on both the math and language arts tests In order 
to fulfill one of the requirements for receiving a regular 
diploma. Students take the test for the first time in the 
ninth grade. 
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Widget Too 



General Information Found In A Typical School District's Report Card, 
1990-91 



Svstem Information 


Grade 
Level 


1988-89 


1989-90 


1990-91 


State 
Averaae 


Number of Schools 


K-12 


5 


5 


5 


na 


Average Daily Membership 


K-12 


3,372 


3,9290 


3,436 


na 


% Student Attendance 


K-12 


95.1 


95.8 


95.6 


na 


% Enrollment Change 


9-12 


-15.2 


-12.1 


-20.1 


na 


%Oversized Classes 


K-12 


2.3 


1.4 


1.5 


na 


%of Students on Free or Reduced Lunches 


K-12 


21.0 


22.0 


23.0 


na 


Expenditure per Pupil 


K-12 


$3,501 


$3,942 


$4,073 


na 


County Per Capita Income 


K-12 


$12,819 


$13,662 


$14,192 


na 


% Elementaiy Schools Accredited by SACS 


K-8 


100 


100 


100 


na 


% Secondaiy Schools Acaedited by SACS 


7-12 


100 


100 


100 


na 


Professional Educator Infomiatlon 




% Professbnais on Career Ladder II and 111 


K-12 


25.6 


28.6 


30.8 


na 


Average Professional Salary 


K-12 


$30,804.37 


$31,590.60 


$33,753.00 


na 


Student Information 




% Diplomas 
Granted 


Regular 


12 


75.8 


73.4 


79.5 


na 


Honors 


12 


20.0 


22.0 


18.6 


na 


Special Education 


12 


1.5 


0.9 


1.0 


na 


Certificate of Attendance 


12 




.09 




na 


Seniors not Receiving 
Diploma in Spring 
Graduation 


12 


2.7 


2.8 


1.0 


na 


% Students in Vo 


catbnal Education Courses 


7-12 


41.0 


41.3 


39.3 


na 


% Students in Special Education 


K-12 


12.1 


12.6 


13.6 


na 


% Chapter 1 Students 


K-12 


12.1 


12.6 


8.7 


na 



Othtr Intomiatlon: 

Parcwit of StudAnt In Attondanc* (%SA). 
This figure shows tt^e average pefoenl of student In 
atterxance daily in your scnooi system for Sto 1 990 
91 year. 

P#rc«ntEnro<lm9nl Chang* (%EC). This 
figure shows the percent change in a group of 
student who started in t\e ninth grade four years 
090 and should have oompleteo the twelfth grade 
this year. It is a feur year average. Decreases 
happen wr>en stixJents drop out of a school, move 
away, graduate early, Call a year, or leave school for 
other reasof^s not listed. 

Percent of Ovsrsiz^d C lai i M (%0C). This 
figure sl'iows the percent of dasses in ail grade 
levels which had waivers for being over the 
maximim dass size. Maximum dass sizes in 
Tenr^ssee are 25 for grades K-3: 28 for grade 4, 30 
for grades 5-6; 35 for grades 7-12: 23 iof vocation. 

P«rc«nt Stud«ntt on FrM or Roduc«d 
Lunch** (%FRL): Students wrxwe family income 
meets certain oriteda are eligible for free or reduced 
price lunches. This figure shows the percent of 
student sin your school system wfx> receive free or 
reduced price lunches. 

Fj(p«ndRur»p«rPiip{|(EPP\ This figure 
shows me averagd number of ddla'S spent for each 
pupil in average daiV attendance for your school 
system. 

County P«rCapfUtncom»(OC ): This figure 
represents the per cap'ta personal income for the 
county in wtiich your school system is -ocated. The 
most recent fiaures avalabte from the U.S. Bureau 
of Economic Analysis are for 1968. 



Porc«nt Elementary/Secondary Schools 
Accroditftd by SACS (%ES): Schools may elect to 
seek acCTeditatfon from the Souther Association of 
College and Schools (SACS) in addition to receiving 
state approval. This agency recognizes quality 
schools, maintains a list of accredited schools and 
requires a continuing school improvement program. 

Percent Professionals on Caro«r ladder 
Levels li and III (%CL): This figure show the 
perceiit of professionaf staff In your achod system 
who have met the standards for Career Levels 11 
a/Tdlll. These are the upper rungs of Tenr>essae's 
Career Ladder program. The number indudes 
regular dassroom teachers, guidance counselors, 
librarians, and administrators. 

Avsrag* Professional Salary (APS) : This 
figure shows the estimated average salary for all 
certificated personnel in your school system. 

Diplomas GrantcKf: These figures shw the 
percent of the twelfth grade dass receivirw different 
types of diplomas. Some school systems nave 
requirement that may exceed these standards. 
Tennessee students may receive four kinds of 
diplomas: 

High School Diploma (D-HS): Awarded to 
students who (a) earn 20 units of credit, (b) make 
passing scores on all components of the 
Profidency Test and (c) are satisfactory records 
of attendance and conduct. 

Honors Diploma (D-HO): School systems 
may offer an optional dpkxna to students who 
meet inaeased requirements established by the 
Stale Board of Education. The requirements 
indude accelerated English, math, sderKe arKl 



soda) students, and a 3.0 grade point average. 

Spocial Education DIpkxna (l>SE): 
Awarded to students vA)0 have satisfacbniy 
completed an individuaiized Education Program 
and who have salsfactDry records of aser)oanoe 
and oor^duct, but who hava notpassed al 
components of the Proficiency Test 

Ccr1Iflcateo(Attsn<knc«(I>CA): Awarded 
to students who hava earned 20 intsof credit 
and who have saHsfactoiy records of attendance 
and conduct, but who fail 10 meet Profidency 
Test standards. 

Studsnts Net RsceMr>a Dtptomt In Spring 
Graduation (D-NR): This n^ure represents 
students vi/ho wiR reoeve they dpbrnas after 
completing sunvDer school or wrio (ailed 13 
complete nigh school. 
Percent or Studtntt ki Vocattonsl Educstton 
CourMM(%VO): This figure shows the percent of 
the school system's avarage daily rnernbership 
enrolled in one or mofft vocational education 
courses. Stidentservoled in mora than ons 
vocatiorLai courses are counted only onoe. 

Percent oe Studtnts ki Speckl Educttion 
(%SE): This HjQMrd show the percent of sudents in 
yoursdiool system who are rooelvingspedal 
education services. 

Petcent o( diepter 1 StudM«t (%CHl): 
Chapter 1 is a federally funded program 10 assist 
students in fy& areas of reading and mathematics. 
This figure shows the percent of student receiving 
services under Chapter 1 . 



accredited SACS, diplomas granted in special education, certificates of attendance 
granted as diplomas, and seniors not receiving diplomas in Spring graduation. 
Appendix A presents the results of this analysis. 

Two correlation procedures were used to generating a response to research 
question #1 . A Pearson Product Moment correlation enabled comparison of variables, 
and Guttman's partial correlation allowed the researchers to develop percentages of 
influence as a means of assessing relationships between independent and dependent 
variables. 

To answer research question #2, the correlations (Pearson and Guttman's) were 
generated for each independent-dopendent variable relationship at each of the four 
defined school levels. 

Research question #3 was answered by computing correlations among 
independent variables. A coefficient of determination (r^) showed the levels of 
interaction between categories (variables). 

Research question #4 required the rank ordering of school districts within the 
sample by system MSO. Comparisons of rankings at all school levels (elementary, 
middle, secondary) could be made. Only the top 10 and bottom 10 districts in the 
rankings are reported. 

Research question #5 required no further statistical analyses. The partial 
correlation coefficients and related percentages of influence previously developed 
provided the necessary data. 

To answer research question #6, changes in MSO upward and downward of one 
standard deviation from school level to school level were first computed using Z-scores 
as the basis for the computation. To further clarify the results, school-level rankings 
were developed. 

A final question was used primarily as a means of focusing conclusions 
and implications. Report cards on schools and the data included in them generate 
poNcy discussions. The findings of this study when added to those of the earlier one 
should be useful to policymakers at all levels. 
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III. FINDINGS 

Findings of tlie study are reported in two ways: (A) a descriptive analysis of tlie 
120 school districts used in the study, and (B) responses to the research questions. 

A. Descriptive Analysis of Scliool Districts 

A profile of the 120 school districts qualifying for inclusion in the study by Report 
Card category was developed. For each category, the report card (state) mean score, 
standard deviation (SD), number of schools submitting data and ranges of scores or 
numbers were compiled. Table 5 presents the profile. 

1. System Information 

All school districts the sample (1 20) reported scores for TCAP and for the TPT. 
When special education students were included in the TPT results, 87.1 percent of all 
students passed the language test, 90.8 percent passed math, and 84.0 passed both. 
When special education students were excluded from the report, 92.2 percent of the 
students across the state passed the language test; 94.9 percent passed the 
mathematics test and 89.7 percent passed both tests. 

Most of the 1 20 school districts studied reported all data for the 20 report card 
categories. One hundred and three reported percentage of oversized classrooms, 48 
reported percentage of elementary schools accredited by SACS, 83 reported 
percentage of secondary schools accredited by SACS, 119 reported percentage of 
professionals on Career Ladder II & III, and 66 reported certificate of diplomas 
awarded. The statewide profile shows approximately 1 3 schools per district with an 
average daily membership of 6,624 students. In 1 990-91 , student attendance averaged 
94.4 percent statewide; enrollments in the districts decreased from the preceding year 
by an average of slightly more than 23 percent. In these districts, approximately 4.4 
percent of all classes exceeded state prescriptions for class size. Almost 42 percent of 
all students state wide received free or reduced lunches. Per pupil expenditures 
averaged $3,442 per district, and county per capita income averaged $12,371. 

2. Professional Educator Information 

Approximately 17 percent of all Tennessee educators had achieved Career 
Ladder Levels II or III by 1990-91 , and average professional salary was $27,465. As 
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few as 6.8 percent of the teachers in a district and as many as 42.5 percent had 
achieved upper Career Ladder status, and average salaries reported ranged from 
$23,262 to $36,505. 

3 Student information 

Eighty percent of all diplomas awarded in the state in 1990-91 were Regular 
diplomas; almost 1 4 percent were Honors diplomas; slightly more than 2 percent were 
Special Education diplomas, and about 1 percent of all students leaving school were 
granted certificates of attendance. More than 6 percent of students graduating did not 
receive their diplomas during spring graduation. 

Almost 48 percent (47.6%) of Tennessee's students were enrolled in vocational education 
classes during the year investigated. Slightly more than 1 6 percent were special education 
students, and another 16 percent were partidpants in Chapter 1 programs. 

4, Comp arison of the 1990-91 Profile with 1988-89 data. 

A few comparisons of data from the 1 990-91 profile (see Table 5) with data from 
1988-89 (see Table 6) are useful. In 1988-89, 76 percent of students taking the TPT 
passed the language test, 90 percent passed mathematics, and 76 percent passed 
both sections. Passing rates for the TPT had risen substantially in language (M=76%, 
92%, respectively), and in passage of both language and mathematics tests by 1991 
(M=76%, 84%, respectively). 

Between 1989 and 1991, average per pupil expenditures had not isen 
much(=$1 00), and average county per capita income had fallen by about $500. 
Average professional salaries of educators had increased about $700. The percentage 
of students receiving free or reduced lunches remained static at approximately 42 
percent, and the number of oversized classes dropped by only 3 tenths of one percent. 

B. Findings Pertinent to Research Questions 

1 . How do school district characteristics currently reported relate to student 
academic achievement results? 
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Table 5. A Report Card Profile of 1 20 Tennessee School Districts Sampled, 
1990-91 data. 



Tennessee Proficiency Test iTPT) 
With Special Education 

Language 
Math 
Both 

Without Special Education 
Language 
Math 
Both 

System Information 

Number of Schools 
Average Dally Membership 
% Student Attendance 
%Enrollment Change 
% Oversized Classes 
% ReecrFtedLcedLirches 

Expenditure Pupil 
County Per Capita Income 
% El. Schools accredited by SACS 
% Sec. Schools accredited by SACS 

Pr ofessional Educator Information 

% Career Ladder II & III 
Average Professional Salary 

Student Infom iatton i% Diplomas Granted) 
Regulai 
Honors 

Spec. Education 
Certlficaie of Attendance 
Seniors not receiving 

Diploma in Spring Grad. 
% Students in Voc. Ed. Classes 
% Students in Special Ed. 
% Chapter 1 Students 











District 


SQ 


Max 


MIn. 


n 


mean 




99 


66 


120 


87,1 


5.8 


100 


68 


120 


90.8 


8.3 


99 


58 


120 


84.0 


5 6 


100 


72 


120 


92.2 


4.6 


100 


74 


120 


94.9 


5.6 


100 


72 


120 


89.7 


20.1 


161 


1 


120 


12.9 


12,415 


103,987 


378 


120 


6,624 


1 


97.4 


91,2 


120 


94.4 




3.6 


-48.3 


120 


-23.0 




23 


0.2 


103 


4.4 




85 


10 


120 


41.7 


$532 


$5,312 


$2591 


120 


$3,442 




$22,097 $8,081 


120 


$12,371 




100 


3 


48 


60.4 




100 


25 


83 


85.3 


6.0 


42.5 


6.8 


119 


16.8 


$2,960 


$36,505$23,262 


120 


$27,465 


9.2 


98.7 


56.3 


120 


80.4 


7.0 


41.7 


1 


102 


13.7 


1.6 


8.6 


0.4 


107 


2.4 


0.7 


2.9 


0.1 


66 


.9 


4.3 


21.3 


0.3 


97 


6.5 


13.7 


98.8 


19.8 


120 


47.6 


3.9 


28.8 


8.2 


120 


16.4 


8.1 


47.5 


2.6 


120 


16.2 
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Table 6. A Report Card Profile of 121 Tennessee School Districts sampled, 1988-89 
data. 



1 21 SCHOOL DISTRICTS 

Report Card 
Sa n Max Min. Mean 



OUTCOMES 

Basic Skills First (BSF) 

Reading 
Math 

Stanford (STAT); Task 2 

Reading 
Math 
English 
Science 
Social Studies 

TN Proficiency Test 

Language 

Math 

Both 

MONEY 

Co./Capita Income ($)(CCI) 
Stud. Expenditure ($)(EPP) 
Aver. Prof. Salary ($) (APS 

SCHOOL SYSTEM 

Average Daily Mem.(#) (ADM) 
Student Attendancer/oSA) 
Oversized Cl?.ss (%0C) 
Fre3/Reducod Lunch (%FRL) 
Career Ladder I I/I 1 1 (%CL) 



4.9 
7.7 



0.5 
0.5 
0.6 
0.5 
0.5 



8.6 
6.4 
9.3 



121 
121 



121 
121 
121 
121 
121 



121 
121 
121 



fPercfint p assing^: 8th grade 
91 65 81 
85 43 66 

(Staninft score): 12th grade 
7 4 5 

6 4 5 

7 4 5 
6 3 5 
6 4 5 

(% Students Passing): 9th grade 



98 
98 
98 



56 
59 
48 



76 
90 
76 



1,962 


121 


19,318 


6,934 


12,878 


509 


121 


4,891 


2,318 


3,304 


2,693 


121 


34,797 


21,286 


26,756 


12,395 


121 


104,788 


375 


5,87 4 


1.3 


121 


97.1 


90.3 


93.6 


4.1 


110 


21.5 


0.1 


3.8 


14.5 


121 


86.0 


9.0 


42.0 


5.9 


121 


41 .5 


4.1 


14.8 



As in the 1988-89 study, a correlation nnatrix (Appendix B) was generated to 
assess the relationship between each reported characteristic and MSOs. However, 
four sets of relationships could be determined for 1 991 : one for Elementary Outcome 
Level (EOL), one for Middle Outcome Level (MOL), one set for High School Outcome 
Level (HOL), and one for the System Outcome Level (SOL). The same correlation 
matrix (see Appendix B) displays relationships between independent variables and 
system outcomes (SOL). 
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In response to question 1, one can see in Appendix B correlations exceeding 
±.50 between four district characteristics and EQL- percent of free or reduced lunches 
(r= -.70), perr-nt of upper career ladder professionals (r= .62), percent of special 
education diplomas (r= -.53), and percent of Chapter 1 students (r= -.68). Five 
characteristics correlated above ±.50 with MOL : percent of free/reduced lunches {r= - 
.69), percent of upper Career Ladder teachers (r= .65), average professional salaries 
(r= .51), percent of Special Education diplomas (r= -.69), and percent of Chapter 1 
students (r= -.69). High correlations (above ±.50 ) existed between HQL and five 
district characteristics: percent of student attendance (r= .53), percent of free/reduced 
lunches (r= -.69), percent of upper Career Ladder teachers (r= .55), percent of special 
education diplomas (r= .55), and percent of Chapter 1 students (r= -.74). When 
academic outcomes (MSG) for the entire system were the focus, four system 
characteristics demonstrated correlations above +.50: percent free/reduced lunches (r= 
-.73), percent of upper Career Ladder teachers {r= .64), percent special education 
diplomas (r= -.62), and percent of Chapter 1 student (r= -.73). Academic outcomes at 
all levels were influenced positively by the presence of expert teachers (upper Career 
Ladder teachers) and to a somewhat lesser degree by attendance. Attendance most 
influenced HOL performance. Most severe negative influences on academic 
performance at all levels were percent of students receiving free/reduced cost lunches 
and percentage of Special Education and/or Chapter 1 students. 

A second set of data relating to question 1 is in Table 7 (also see Appendix C) 
presented a Guttman's Partial Correlation matrix for each of the four outcome levels 
and for 15 targeted system characteristics, and a display of the percentage of influence 
exerted by each system cnaracteristic on each set of MSOs. Some findings produced 
from these analyses included: 

1 . The system characteristics having greatest imp act on student academic 
performance were not ie same at all levels (see Figure 1 ). The factor most 
influencing the EOL was per pupil expenditure (11 .2%). Middle school student 
academic performance was most impacted by the same factor (8.1%). 
Academic performance among high school students were generally reported by 
their attendance (13.6%), as has overall academic performance in the school 
district (13.3%). 

2. The factor having lea.qt impact on MSOs also varied by school level (see Figure 
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±.50 between four district characteristics and EOL : percent of free or reduced lunches 
(r= -.70), percent of upper career ladder professionals (r= .62), percent of special 
education diplomas (r= -.53), and percent of Chapter 1 students (r= -.68). Five 
characteristics correlated above ±.50 with MOL : percent of free/reduced lunches (r= - 
.69), percent of upper Career Ladder teachers {r= .65), average professional salaries 
(r= .51), percent of Special Education diplomas (r= -.69), and percent of Chapter 1 
students (r= -.69). High correlations (above ±.50 ) existed between HOL and five 
district characteristics: percent of student attendance (r= .53), percent of free/reduced 
lunches (r= -.69), percent of upper Career Ladder teachers (r= .55), percent of special 
education diplomas (r= .55), and percent of Chapter 1 students (r= -.74). When 
academic outcomes (MSO) for the entire system were the focus, four system 
characteristics demonstrated correlations above +.50: percent free/reduced lunches (r= 
-.73), percent of upper Career Ladder teachers (r= .64), percent special education 
diplomas (r= -.62), and percent of Chapter 1 student (r= -.73). Academic outcomes at 
all levels were influenced positively by the presence of expert teachers (upper Career 
Ladder teachers) and to a somewhat lesser degree by attendance. Attendance most 
influenced HOL performance. Most severe negative influences on academic 
performance at all levels were percent of students receiving free/reduced cost lunches 
and percentage of Special Education and/or Chapter 1 students. 

A second set of data relating to question 1 is in Table 7 (also see Appendix C) 
presented a Guttman's Partial Correlation matrix for each of the four outcome levels 
and for 1 5 targeted system characteristics, and a display of the percentage of influence 
exerted by each system characteristic on each set of MSOs. Some findings produced 
from these analyses included: 

1 . The system characteristics having greatest impact on stud ent academic 
performance were not the same at all levels (see Figure 1 ). The factor most 
influencing the EOL was per pupil expenditure (11 .2%). Middle school student 
academic performance was most impacted by the same factor (8.1%). 
Academic performance among high school students were generally reported by 
their attendance (13.6%), as has overall academic performance in the school 
district (13.3%). 
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Table 7 



Guttman's partial correlation used to evaluate the 15reprot card categories from 4 
educational levels (elementary (EOL), middle schoo (MOL)I, high schoo (HOL)I, and 
system (SOL), 1990-91 Tennessee school district report card data. 



2. 


r 
o 

C/J 


ADM 


vs% 


O 
UJ 


%0C 


tr 

LL 


EPP 


CC! 


%CL 


APS 


D-HS 


D-HO 


0A% 


%SE 


%CH! 


EOL 


0.2 


0.1 


6.7 


0.3 


3.4 


7.3 


11.2 


0.4 


3.2 


1.0 


1.1 


0.3 


0.8 


1.5 


2.1 


MOL 


0.0 


0.2 


5.9 


0.3 


2.8 


2.3 


8.1 


0.0 


4.9 


0.3 


1.9 


1.5 


1.0 


0.1 


6.0 


HOL 


4.5 


5.3 


13.6 


3.5 


0.2 


0.3 


0.4 


0.6 


0.0 


2.7 


0.2 


0.4 


4.5 


0.0 


4.7 


SOL 


0.4 


0.9 


13.3 


1.5 


3.1 


4.7 


9.4 


0.4 


3.1 


0.1 


1.5 


0.2 


2.9 


0.2 . 


6.5 



o 



The factor having least impact on MSOs also varied by school level (see Figure 
2). The size of the system (ADM) had least influence on elementary student 
performance (0.1 %). Neither the number of schools in a system nor the county 
per capita income had any influence on MOLs (0.0%). HOL was least Influenced 
by the percentage of Special Education students in the district and the 
percentage of Career Ladder II and III teachers teaching there (0.0%). Overall 
MSO in a system was least impacted by average professional salaries of 
educators (0.1%). 

3. Percentage of oversized classes, a rough indicator of the influence of class size 
on student performance, has increasingly less influence on student academic 
performance as students progress from elementary to middle to high school. 
Even at its most influential point in the educational continuum (the elementary 
years), this factor accounts for only 3.4 percent of whatever it is that influences 
student academic outcomes. 



2. Are there differences in the relationships between dependent and 
independent variables at different school levels? 

The answer to this question is clearly "yes" as demonstrated in Appendix 0. We 
have already reported the differences in system characteristics having most and least 
impact on student acddemic outcomes at the various school levels. Nq system 
characteristic influences student academic outcomes in the same way at all 
school/district levels. Further, the combined set of 15 characteristics does not exert the 
same amount of influence MSOs at any of the four levels studied. This finding will be 
explored more completely in response to research question 5. 
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In addition to the relationships demonstrated in Appendix C that have already 
been reported, a few others are important. The presence of upper Career Ladder 
teachers appears to have greatest impact on student performance at the middle school 
level (4.9%). The average professional salaries paid within a school district do not have 
great influence on student performance, but they have more influence (2.7%) on 
secondary students than on any other group. The socio-economic level of the 
community (county per Capita income) had less than 1 percent influence on academic 
outcomes at any level. 

The line graph presented in Figure 1 simply reinforces the statistics presented in 
the accompanying Appendix C. Note particularly the positions of the influence occupied 
by percent student attendance expenditure per pupil and percentage of students 
receiving free/reduced lunches in the relationships to the positions of other variables. 

Further, note the bar graph presented in Figure 2 when the percentages of 
influence for the three levels were summed. High school outcomes do not appear to be 
influenced by percent of oversized classes, percent of free/reduced lunches or funds 
spent on student (expenditure per pupil). Middle school outcomes are not really 
influenced by the district's size (number of schools / average daily membership), 
percent of enrollment change, county per capita income, average professional salary, or 
the percentage of in the Special Education classes. Finally, the district's size, 
enrollment change, or socio-economic factors (County per Capita Income) does not 
influence elementary student outcomes. 

3. How do reported schooi characteristic relate to each other? 

The answer to this question is found in Appendix B. The correlation matrix 
reveals eight correlations exceeding ±.50. The relationship between number of schools 
in the system and student attendance is strongly negative (r=: -.54). The same can be 
said of the relationships between student attendance and size of school district (n= - 
.54) and between percentage of student receiving free/reduced lunches and 
attendance (r=: -.54). None are surprising statistics. 

There is a strong positive correlation (r=: .53) between percentage of special 
education diplomas awarded in a district and the percentage of students receiving 
free/reduced cost lunches. A strong positive correlation (r= .78) exists between 
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Figure 1 The percentage of influence for the 15 report card categories and the four levels 
(elementary [EOL), middle [MOL], high school [HOL], and system (SOL], 1990-91 
Tennesse report card data. 
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Figure 2 The percentage of influence for the 15 categories and the three educational levels: 
elementary (EOL), middle (MOL), and high school (HOL). 
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percentage of Chapter 1 students in a school district and percentage of students 
receiving free/reduced cost lunches. Special education, free/reduced meals, and 
Chapter 1 are closely linked. 

The relationship between average professional salaries in a system and 
expenditure per pupil is strongly positive (r= .79). Communities that spend more on 
education pay their teachers and administrators better than do other communities. For 
example, the very large positive correlation (r=:.99) between number of schools and size 
of school district does not tell us much. A strong positive correlation {r=.51 ) is found 
between percentage of students receiving special education students and percentage 
of students not receiving diplomas in the spring. 

The eighth correlation exceeding .50, this one a positive correlation (r=.50) is 
between percentage of special education diplomas awarded and percentage of 
students enrolled in vocational education programs. This correlation could reflect the 
creation of vocationally-oriented programs for special education students, or placement 
of special education students in vocational programs, regardless of the suitability of the 
programs to the students. 

4. When rank ordered on the basis of student outcomes, how do school 
districts within the state perform in terms of reported school and 
community characteristics? 

To explore this comples question, the investigators generated rankings by MSG 
at the four levels being investigated and by system characteristics for the top 1 0 and 
bottom 10 producing systmes, using system MSOs (SOL) as the anchor. Table 8 and 
Appendix D present the findings. 

Table 8 displays the relationships between SOLs and at school levels. Among 
important findings are the following: 

1 . The system having the highest MSO (#72) had the highest elementary and 
middle school MSOs, but not the highest high school MSO. 

2. Eight of the top 1 0 systems ranked by district MSO were not in the top 1 0 at the 
elementary, middle, or high school levels. 

3. The district ranking 10 SOL ranked 60th in high school student performance. 

4. No district ranking among the bottom 1 0 districts in district MSO ranked above 
94th at any school level. 
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Table 8 Profiles of Schoc; Dsitrict Rankings By Student Academic Performance, 
1990-91 data. 





Elementary 


Middle 


High School 


System 


Differences 




SCH 


EOL 


Rk 


MOL 


Rk 


HOL 


Rk 


SOL 


Rk 
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DIff. 


# 
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1.61 
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1.78 
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1.55 
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1 1 U 


1 71 


7 


1 1Q 


12 


1.42 


6.5 


1.44 


7 


l!71 


1.19 






1.58 


13 


1.82 


7 


0.85 


19 


1.42 


8 


1.82 


0.85 






1.71 


7 


1.18 


13 


1.29 


10 


1.39 


9 


1.71 


1.18 


0.53 


29 


2.22 


2 


1.83 


6 


-0.03 




1.34 


10 


2.22 


-0.03 


2.25 












Bottom 10 Svstems 










97 


-0.75 


94 


-1.09 


112 


-0.75 


106 


-0.86 


111 


-0.75 


-1.09 


0.34 


16 


-0.88 


10S.5 


-0.43 


98 


-1.36 


115 


-0.89 


112 


-0.43 


-1.36 


0.93 


62 


-1.40 


115.5 


-0.60 


107 


-0.82 


108 


-0.94 


113 


-0.60 


-1.40 


0.80 


46 


-0.88 


106.5 


-1.25 


115 


-1.02 


112 


-1.05 


114 


-0.88 


-1.25 


0.37 


10 


-1.14 


113 


-1.72 


117 


-0.90 


109 


-1.25 


115 


-0.90 


-1.72 


0.81 


58 


-0.88 


106.5 


-1.09 


113 


-1.82 


117 


-1.26 


116 


-0.88 


-1.82 


0.94 


78 


-1.79 


117 


-1.25 


114 


-1.43 


116 


-1.49 


117 


-1.25 


-1.79 


0.53 


41 


-1.79 


118 
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119 
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107 
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1.78 
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-2.43 
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-2.54 


120 


-2.72 


120 


-2.56 
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-2.43 


-2.72 


0.29 


30 


-3.21 


120 


-2.52 


118 


-2.70 


119 


-2.81 


120 


-2.52 


-3.21 


0.69 



Appendix D provides data about school district/community characteristics in 
relation to system level MSO rankings. It also profiles the relationships between 
system/community factors and HOLs. Note the following: 

1 . There are no readily identifiable patterns of school/connnnunity characteristics 
among those currently reported that produce high achieving or low achieving 
school systems. 

2. There are no common patterns of school/community characteristics among 
those reported that appear consistently to produce high achievement or low 
achievement among nigh school students. 
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3. Typical biases about characteristics necessary in a system or community to 
produce high achievement (e.g., money, larger or smaller schools, small 
classes) are not confirmed by the data available. Schools and communities with 
a range of the characteristics currently reported produce both higher and lower 
academic achievement. 

5. Do the reported school district characteristics appear to represent ail or 
most factors that influence student academiL achievement? 

The answer to this question is found in Appendix C. Clearly, the answer is, 

"NO." Together, the 1 5 characteristics under investigation provide 39.6 percent of the 

influence on EOL, 35.3 percent of the influence on MOL, 40.9 percent of whatever 

influences HOL, and 48.2 percent of the influence on SOL. These factors influence 

outcomes at different levels in different ways, and together they account for less than 

half of whatever influences student performance at any level. 

6. Is there evidence of major change in student academic performance from 
one school level to another withm school districts? 

Table 9 provides the data pertinent to this question. Eleven systems 
demonstrated shifts downward in MSO of at least one standard deviation somewhere 
between the elementary and the high school levels. Sometimes the shift occurred from 
elementary to middle school, sometimes from middle to high school. Sometimes the 
change was continuous from level to level, and sometimes a dramatic shift occurred 
from elementary to middle, but began to reverse from middle to high school. 

Twelve systems demonstrated changes of at least one standard deviation 
upward over the three school levels. Again the patterns of change were not always 
constant, and the shifts occurred at different points in different systems. 

Some of the notable change patterns can be seen in reviewing the changes in 
academic rankings of the system from level to level: 

1 , Six of the 1 1 systems showing downward shifts in MSO had consistent 
downward trends from the elementary to middle to high school levels. 

2, Three districts showed significant declines in MSO from the elementary to 
middle school level, but reversed the trend from middle to high school. System 
#82 demonstrated a dramatic downward shift from elementary to middle school 
{20th to 78th) and a dramatic shift upward from middle to high school (78th to 
18th). 
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Table 9 School districts with outomes greater/smaller than +<1 .0 z-scores between the 
elementary, middle, or high school levels. 
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3. Two districts (#71 , #9) displayed better student performance (by rank) at tiie 
middle school level than at the elementary level, but dropped markedly in the 
high school rankings. 

4. Of the 12 systems demonstrating upward shifts in MSO, 6 showed consistent 
patterns of improvement at each school level. Perhaps the most dramatic 
pattern was exhibited by system #1 which ranked 106 (of 120) in EOL, 23 in 
MOL and first in HOL Data for this system also clearly point up the limited value 
of district level rankings. In the composite, this system ranked 28th in SOL. 

5. Three systems (#41 , #74, #52) displayed downward patterns of achievement 
from elementary to middle school, but strong upward patterns from middle to 
high school. 

6. Three systems (#90, #64, #51 ) showed strong upward trdnds in MSO and 
ranking from the elementary to middle school level, bul reversed the pattern from 
the middle to the secondary level. 

The causes of the changes found among these 23 school districts are unknown. 
They could relate to the quality of instruction students received at the several levels. 
They could reflect an emphasis on "teaching to the test" at certain levels. They could 
indicate the lack of alignment between outcome measure (tests) and curriculum. They 
might be caused, in part, by the movement to a new set of tests (TCAP) during the year 
being investigated. What is clear is that outcome data and rankings reported at the 
system level have limited utility in identifying what is happening academically within a 
system or in targeting areas for improvement. 



IV. CONCLUSIONS AND IMPLICATION OF THE STUDY 



As in the initial study, investigators framed a final research question as a means 
of developing useful conclusions and implications. 

10. How might the findings of this study inform educational policy at state and 
local levels? 

Several of the conclusions the 1 988-89 studv were reinforced by the results ot 
the 1990-91 investigation. Specifically, policymakers at all levels need to consider that 
few of the individual inputs commonly associated in people's minds with production of 
student achievement have much impact on student performance. With the exception of 
student attendance (and perhaps per pupil expenditure) treatment of any isolated 
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variable will have little effect. If we want to improve or change student performancQ. a 
systemic approach to education change is an absolute necessitv. 

In the 1 988-89 study, the researchers concluded that the eight system 
characteristics taken from the Tennessee Report Cards for analysis were of limited 
value; i.e., they gave limited information to policymakers and educators who want to 
improve education in their states and local communities, because these variables 
accounted for so little of the influence on student outcomes. In the 1990-91 study, 15 
variables were available for examination. Again, they do not appear to be the "right 
ones," i.e., they don't tell us enough about what influences student achievement. 
Based on the two studies, knowledge gained from review of related research and 
experience in schools, the investigators urge that consideration be gi ven to collecting. 
re p orting, and analvzing data on organization culture, student motivation, parental 
involvement, instructional methodologies, curriculum feature s and other factors that 
may have significant influence on student performance. 

When reviewing the results of the 1988-89 study, the investigators suggested 
that building-level data are probably more useful and more valid than distrint^lQvel data 
for use in report cards. That conclusion is confirmed by the present study. Major 
variations and fluctuations in results appeared from school level to school level within 
individual school districts. Identification of sources of these differences could be useful 
to educators and policymakers seeking improvement. Even the 1990-91 study did not 
have building-level data available for analysis. School-level data may reflect conditions 
across several schools. 

Report cards are on ly as good as the ass essments used to determine student 
performance . The 1988-89 study raised some questions about the assessments being 
used. Those questions are highlighted in the findings of the current study. Enough 
variations in similar analyses from study one to study two exist to suggest that the 
differences in student outcome measures are probably one cause. 

Finally, "What is the purpose of School District Report Cards?" The question is 
not an antagonistic one, but a supportive one. Definition of purpose or purposes is 
central to assessing the value of report card contents. A recent editorial in the 
Nashville Tennessean (1992) speaks of Tennessee's report cards in glowing terms: 
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It (the Report Card) is simply the most comprehensive report in this 
or any state on school funding and student performance. . . 

The reports are more than just a tool for comparison, however; they 
can empower local communities to act. The reports give Tennesseans the 
power to get the job dona and make the grade for better schools. {p.40). 



If the purpose of the Tennessee Report Card is simply to report the status of a 
community's schools and selected factors generally associated with them, the current 
report card does that reasonably well. If the purpose is to provide citizens, parents, 
educators and policymakers meaningful information upon which to make decisions for 
improvement, much is lacking. At least 50 percent of what influences student 
performance has not bee reported. This can provide serious impediments to school 
improvement if education leaders focus on what is reported as the way to raise their 
schools in the rankings, rather than focusing upon the non-reported, but more 
important, factors. 
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Appendix A 



Results of Kaiser Test of Variable Sampling Adequacy 

20 report card variables 
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Appendix C 



Partial Correlations and Percent of Influence of 15 School District 
Characteristics On Mean Student Outcomes 
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Appendix D 



Profiles of School District Rankings by District Mean Student Outcomes 
and School District Community Characteristics 
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